Log-structured storage developed in order to provide a more efficient means for storing data in persistent storage devices. Data and metadata changes are sequentially recorded as log records in a log structure reducing the number of operations to persist the data and metadata changes. For systems that frequently add and/or modify data, such as database systems, log-structured storage reduces the latency for recording new data as well as modifying data already stored. Log records are typically stored in storage according to a log record sequence so that log records that are dependent upon prior log records may be processed in a correct order.
Situating log-structured storage systems in a distributed system architecture may introduce various complications that blunt the efficiency of log-structured storage. For example, distributed storage systems may have to ensure consistency across multiple systems or devices. Various different schemes may be employed to ensure consistency. However, ensuring consistency may increase the cost and/or time to record new log records at the log-structured storage. For example, synchronous approaches may require that each log record be persisted in the distributed storage system and acknowledged back to a storage client before a next log record may be sent to be recorded. Alternatively, asynchronous approaches may allow for other log records to be sent and recorded while prior log records have not yet been recorded. While changes to a log processed synchronously may be easily ordered when received at a storage node of a log-structured distributed storage system if ordered according to the log record sequence, asynchronous updates may prove more challenging. Acknowledgments of log records may be delayed until prior log records are received, or log records may be stored out of order, which burdens the storage node when it is time to perform various operations dependent on the ordering of log records.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that the embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). The words “include,” “including,” and “includes” indicate open-ended relationships and therefore mean including, but not limited to. Similarly, the words “have,” “having,” and “has” also indicate open-ended relationships, and thus mean having, but not limited to. The terms “first,” “second,” “third,” and so forth as used herein are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless such an ordering is otherwise explicitly indicated.
Various components may be described as “configured to” perform a task or tasks. In such contexts, “configured to” is a broad recitation generally meaning “having structure that” performs the task or tasks during operation. As such, the component can be configured to perform the task even when the component is not currently performing that task (e.g., a computer system may be configured to perform operations even when the operations are not currently being performed). In some contexts, “configured to” may be a broad recitation of structure generally meaning “having circuitry that” performs the task or tasks during operation. As such, the component can be configured to perform the task even when the component is not currently on. In general, the circuitry that forms the structure corresponding to “configured to” may include hardware circuits.
Various components may be described as performing a task or tasks, for convenience in the description. Such descriptions should be interpreted as including the phrase “configured to.” Reciting a component that is configured to perform one or more tasks is expressly intended not to invoke 35 U.S.C. §112, paragraph six, interpretation for that component.
“Based On.” As used herein, this term is used to describe one or more factors that affect a determination. This term does not foreclose additional factors that may affect a determination. That is, a determination may be solely based on those factors or based, at least in part, on those factors. Consider the phrase “determine A based on B.” While B may be a factor that affects the determination of A, such a phrase does not foreclose the determination of A from also being based on C. In other instances, A may be determined based solely on B.
The scope of the present disclosure includes any feature or combination of features disclosed herein (either explicitly or implicitly), or any generalization thereof, whether or not it mitigates any or all of the problems addressed herein. Accordingly, new claims may be formulated during prosecution of this application (or an application claiming priority thereto) to any such combination of features. In particular, with reference to the appended claims, features from dependent claims may be combined with those of the independent claims and features from respective independent claims may be combined in any appropriate manner and not merely in the specific combinations enumerated in the appended claims.